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Too  many  projects  suffer  from  unachievable  budget  and  schedule  goals, 
caused  by  unrealistic  estimates  and  the  failure  to  quantify  and  communicate 
the  uncertainty  of  these  estimates  to  managers  and  sponsoring  executives. 
Some  projects  are  beginning  to  use  Monte  Carlo  methods  to  improve  cost 
and  schedule  estimating;  however,  it  is  not  always  done  in  a  systematic  manner, 
mutually  understood  by  both  customer  and  supplier.  This  article  provides  a 
systematic  approach  based  on  quantification  of  expertjudgment  in  a  pragmatic 
and  consistent  fashion. 


Projects  often  fail  to  predict  or  ac¬ 
commodate  the  risk  and  uncertainty 
of  budgets  and  schedules  in  a  rigor¬ 
ous  or  structured  manner.  This  is  of  par¬ 
ticular  consequence  in  the  “better,  faster, 
cheaper”  era  of  program  management.  The 
risk  is  seldom  quantified  in  a  manner  that 
the  estimators,  the  management  hierarchy, 
and  the  customer  mutually  understand, 
accept,  and  acknowledge.  Furthermore, 
projects  are  often  planned  to  “best-case” 
even  though  few  of  the  participants, 
particularly  the  managers  and  sponsoring 
executives,  recognize  and  understand  the 
implications  of  a  best-case,  “green-light” 
plan.  Although  it  may  indeed  be  prudent 
to  work  to  such  a  best-case  plan,  managers 
and  executives  fail  to  concede  that  the 
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probability  of  achieving  the  “best-case” 
goal  is  by  its  very  nature  zero1 — and  that 
faster  and  cheaper  is  also  riskier.  This 
failure  inevitably  results  in  misunder¬ 
standings  and  unrealistic  goals. 

Some  projects,  as  a  result  of  such 
unrealistic  goals  and  with  pressure  from 
an  uninformed  customer,  are  driven  to 
establish  harsh  policies  that  serve  to 
undermine  the  very  objectives  they  hope 
to  achieve.  Methodical  and  rigorous  quan¬ 
tification  of  the  uncertainty  of  cost  and 
schedule  estimates  is  key  to  interpreting, 
and  realistically  managing  and  mitigating, 
cost  and  schedule  risk.  This,  in  the  end, 
will  lead  to  improvement  of  overall  project 
management. 
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Expert  judgment  is  typically  the  crux 
of  cost  and  schedule  estimates,  but  in  the 
spectrum  of  the  risk  management  pro¬ 
cess,  quantifi¬ 
cation  of  expert 
judgment  is  the 
weakest  area. 
Transitioning 
from  English 
language  state¬ 
ments  of  experts 
to  the  mathe¬ 
matical  expres¬ 
sions  required 
by  analytical 
tools  is  done  inconsistently  if  done  at  all. 
Professional  program  planners  are  begin¬ 
ning  to  apply  statistical  methods  to  sched¬ 
ule  and  cost  analysis  in  an  attempt  to  deal 
with  this  problem. 

This  article  offers  a  methodology  for 
statistically  quantifying  the  risk  to  cost  and 
schedule  resulting  from  the  uncertainty2 
of  the  estimates  that  underlie  any  cost- 
schedule  analysis.  Some  simple  rules  for 
cost-schedule  risk  mitigation  are  postu¬ 
lated  that  provide  a  structured  focus  for 
the  methodology  offered.  These  rules 
provide  the  expert  with  a  logic  basis  that 
is  fundamental  to  consistent  and  sensible 
quantification  of  the  risk  elements.  These 
rules  are: 


•  Plan  “best  case”  and  preclude  imple¬ 
menting  a  self-fulfilling  prophesy. 

•  Budget  “most  likely”  and  recognize 
real-world  risk  and  uncertainty. 

•  Protect  for  “worst  case”  and  acknowledge 
the  conceivable. 


Crucial  to  implementation  of  these  rules 
is  the  credibility  of  the  best-case  estimates. 
They  must  be  honest  and  truly  achievable 
in  the  best-case  scenario;  unrealistic  targets 
cannot  drive  them.  Only  then  can  a  credible 
quantification  of  risk  be  applied.  On  this 
basis,  a  specific  approach  to  interpreting 
expert  judgment  and  quantifying  cost  and 
schedule  risk  is  offered. 


Statistical  Analysis 


The  term  “risk”  implies  a  stochastic 
(probabilistic)  process,  and  quantification 
requires  a  model.  Such  a  model  can  be 
developed  by  ascribing  to  each  element 
of  the  cost  estimate,  or  the  schedule  pro¬ 
jection,  a  probability  distribution  func¬ 
tion  (PDF)  representing  the  likelihood 
of  completing  the  particular  cost  element 
or  scheduled  task  at  a  specific  cost  or  for 
a  specific  duration.  Monte  Carlo  simula¬ 
tion  techniques  can  then  be  applied  to  the 
model  to  forecast  the  entire  range  of 
possible  end  results. 

A  simple  triangular  distribution  is  a 
reasonable  PDF  for  describing  the  risk  or 
uncertainty  for  a  cost  element  or  task 
duration  estimate.  Its  structure  is  based  on 
the  minimum  possible  cost  and  duration 
(plan  best  case),  the  most  likely  cost  and 
duration  (budget  most  likely)  and  the 
maximum  possible  cost  and  duration 
(protect  for  worst  case)  as  illustrated  in 
Figure  1. 

The  height  of  the  triangle  is  2/(max-min) 
such  that  its  area  is  unity.  The  parameters 
are  simple,  intuitively  easy  to  compre¬ 
hend,  and  amenable  to  a  mathematical 
formulation  compatible  with  cost  and 
schedule  models  and  fast  Monte  Carlo 
analysis.  Other  more  complex  distributions 
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FigLrel.  Risk:  Unoertairty  far  Cost  Elerrertt  or  Task  Duration 


could  be  used  such  as  the  Beta  or  Weibull,  distribution  is  lost.  The  triangular  PDF  will 
but  little  if  anything  is  gained,3  and  the  form  the  basis  of  the  quantification  of  risk 
intuitive  simplicity  of  the  triangular  offered  here. 


Table  1.  Risk  Fader  Multipliers 


Code 

Min 

Most  Likely 

Max 

Low 

L 

i 

1.04 

1.10 

Low+ 

L+ 

i 

1.06 

1.15 

Moderate 

M 

i 

1.09 

1.24 

Moderate  + 

M  + 

i 

1.14 

1.36 

High 

H 

1 

1.20 

1.55 

High  + 

H  + 

i 

1.30 

1.85 

Very  high 

V 

i 

1.46 

2.30 

Very  high+ 

V+ 

1 

1.68 

3.00 
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Quantifying  Judgment 


The  “expert”  is  first  asked  to  provide  a 
green-light,  best-case  estimate  of  each 
cost  or  schedule  element  in  the  model,  and 
then  is  asked  to  provide  an  assessment  of 
risk  (his  or  her  unsureness)  associated  with 
the  estimate  for  each  element.  Table  1  relates 
the  risk  adjectives  of  low,  moderate,  high,  and 
very  high  to  a  set  of  risk  factor  multipliers. 

The  table  has  been  developed  to  facili¬ 
tate  and  guide  the  expert  in  risk  estima¬ 
tion.  These  factors,  when  multiplied  by  the 
best-case  estimate,  are  the  parameters  of 
the  triangular  PDF  for  a  particular  element 
of  cost  or  schedule  based  on  its  associated 
risk. 

This  table  is  intended  to  provide  a  rea¬ 
sonable  range  of  risk  quantification  that 
fits  well  with  the  perceptions  and  experi¬ 
ence  of  engineers  and  estimators.  It  is 
based  on  the  author’s  experience  as  a 
project  manager,  planner,  proposal  evalu¬ 
ator,  and  as  an 
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expert  provid¬ 
ing  cost  and 
schedule  esti¬ 
mates  for  many 
years  and  on 
many  projects. 
The  table  has 
also  been  used 
by  engineers  at 
the  Naval  Re¬ 
search  Labora¬ 


tory  to  develop 
cost  and  schedule  estimates  for  spacecraft 
projects.  The  engineers  adapted  readily  to 
the  table  and  were  content  with  the  adjec¬ 
tives  and  the  range  of  the  associated  mul¬ 
tipliers.  The  author  is  unaware  of  any  other 
reference  that  would  serve  to  substantiate 


(or  refute)  Table  1.  It  is  suggested  that  an 
approved  standard  for  these  multipliers  be 
developed  that  is  generally  accepted  and 
consistently  applied  industry-wide.  Table 
1  can  be  used  in  the  interim  and  can  serve 
as  a  point  of  departure  for  a  more  widely 
accepted  standard. 

These  particular  risk  factor  multipliers 
are  arithmetically  derived  from  a  few  basic 
assumptions,  which  provide  a  structure 
and  logic  to  the  risk-factor  multipliers  that 
may  facilitate  the  debate  and  development 
of  a  standard.  They  are  as  follows: 

•  For  an  element  of  a  cost  or  schedule  es¬ 
timate  coded  low  risk,  “max”  (worst-case 
performance)  is  defined  as  10%  greater 
than  “min”  (best-case  performance) 
and  “most  likely”  is  defined  as  4% 
greater  than  “min.”  This  reasonable 
premise  for  a  low-risk  element  provides 
a  base  for  the  derivation.  • 

•  The  “most  likely”  set  is  then  derived 
as  a  geometric  progression  of  the  per¬ 
cent  increase  for  the  eight  risk  codes 
with  a  common  ratio  of  1.5  (e.g.,  6%  = 
1.5  x  4%,  9%  =  1.5  x  6%,  and  so  forth 
as  the  risk  increases  from  low  to  very 
high). 

•  For  an  element  coded  as  “very  high+,” 
“max”  is  capped  at  200%  greater  than 
“min.”  The  rationale  for  this  cap  is  that 
no  prudent  manager  would  allow  a 
growth  beyond  three  times  plan  with¬ 
out  intervention  and  mitigating  action. 

•  The  common  ratio  (1.534)  for  the 
“max”  geometric  progression  is  then 
derived  from  the  first  and  last  element 
of  the  set  (e.g.  10%  and  200%). 
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Figure  2.  Risk  Factors  Graphically  Illustrated 


The  multiplier  factors  associated  with 
the  hierarchy  of  risk  approximate  a  geo¬ 
metric  progression.  Therefore,  as  the  risk 
increases,  the  probability  distribution  be¬ 
comes  more  asymmetrical  on  the  “max” 
side,  as  we  intuitively  would  expect.  An 
additional  degree  of  flexibility  is  provided 
by  the  inclusion  of  the  “+”  categories. 

These  factors,  illustrated  graphically  in 
Figure  2,  provide  a  convenience  to  the  esti¬ 
mator;  their  use  will  provide  a  degree  of 
consistency  from  estimator  to  estimator 
and  from  estimate  to  estimate. 

Risk  Factor  Attributes _ 


Below,  typical  attributes  are  suggested 
that  characterize  the  risk  factors.  These 
attributes  illustrate  situations  that  could 
be  a  basis  and  substantiation  for  a  particu¬ 
lar  risk  evaluation.  Risk  factor  attributes 
can  be  tailored  for  a  particular  project  and 
perhaps  they  could  be  expanded  as  part 
of  establishing  an  approved  standard; 


however,  care  should  be  taken  not  to  de¬ 
velop  rigorous  cookbook  bureaucratic 
tests.  We  must  accept  that  in  the  end  all 
estimates  are  judgments,  hopefully  made 
by  experienced  individuals  who  are  moti¬ 
vated  and  unfettered  in  their  task.  Most 
expert  evaluators  will  rely  on  their  own 
experience  as  guided  by  Table  1. 

Low-Risk  Attributes 

As  applied  to  design  tasks.  Existing 
proven  designs  are  used  extensively; 
requirements  are  well  defined  and  readily 
achieved;  development  effort  is  minimal; 
and  an  innovative  approach  materially 
simplifies  design  implementation. 

As  applied  to  production.  Extensive 
use  is  made  of  proven  hardware  or  soft¬ 
ware  produced  by  previous  suppliers; 
exotic  processes  and  tooling  are  not  re¬ 
quired  for  production;  materials  and  parts 
are  readily  available;  and  an  innovative 
approach  materially  simplifies  production. 

As  applied  to  test  and  verification. 
Extensive  use  is  made  of  proven  hardware 
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and  software  produced  by  previous  sup¬ 
pliers;  alignments  and  calibrations  are  not 
critical;  test  tools  and  equipment  are 
readily  available;  and  an  innovative 
approach  materially  simplifies  testing; 
performance  demands  are  reasonable  and 
have  been  realistically  suballocated; 
design  provides  significant  margin  above 
the  requirement;  achievement  of  the  design 
margin  is  not  precipitously  crucial  to  mis¬ 
sion  success;  and  an  innovative  approach 
materially  simplifies  accomplishment  of 
the  mission  requirement. 


Very  High  Risk  Attributes 

As  applied  to  design.  Extensive  use 
is  made  of  new  and  unproven  designs; 
requirements  are  poorly  defined  and 
unlikely  to  be  achieved;  development  effort 
is  extensive;  and  an  “innovative”  approach 
materially  complicates  the  design. 

As  applied  to  production.  Extensive 
use  is  made  of  unproved  hardware  or  soft¬ 
ware  never  previously  produced;  many 
exotic  processes  and  undeveloped  exotic 

tooling  are  es¬ 
sential  for  suc¬ 
cessful  produc¬ 
tion;  materials 
and  parts  are 
not  in  produc¬ 
tion,  require  de¬ 
velopment,  are 
in  short  supply, 
or  otherwise  are 
not  currently 
available  through  normal  vendor  pro¬ 
curement;  an  “innovative”  approach 
materially  complicates  production. 

As  applied  to  test  and  verification. 
Extensive  use  is  made  of  unproved  hard¬ 
ware  or  software  never  previously  pro¬ 
duced;  most  alignments  and  calibrations 
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are  difficult  and  critical  to  performance; 
exotic  test  tools  and  equipment  are  essen¬ 
tial,  not  readily  available,  and  require 
development;  designs  result  in  unstable 
platforms,  timing  interfaces,  and  electri¬ 
cal  outputs;  an  “innovative”  approach 
materially  complicates  testing;  the  allocated 
performance  budget  is  unrealistic;  the 
design  provides  no  significant  margin  above 
the  requirement;  achievement  of  the  design 
margin  is  precipitously  crucial  to  mission 
success;  and  an  innovative  approach  mate¬ 
rially  complicates  accomplishment  of  the 
mission  requirement. 

Moderate  and  High-Risk  Attributes 

A  grade  of  “moderate”  or  “high”  is 
based  on  the  evaluator’s  judgment,  con¬ 
sidering  the  risk  extremes  as  defined  for 
“low-risk”  and  “very  high  risk.”  Attributes 
will  range  from  modification  of  existing 
design  or  catalog  design  to  new  designs 
and  high  technology. 


Interpreting  the  Results 


The  composite  results  of  a  Monte  Carlo 
analysis  of  a  series  of  cost  elements  in  a  hypo¬ 
thetical  project  will  be  a  probability  fore¬ 
cast  such  as  that  illustrated  in  Figure  3.  In 
concert  with  the  risk  mitigation  rules  cited 
at  the  beginning  of  this  article,  the  project 
manager  would  initially  allocate  a  budget 
to  his  project  elements  totaling  about  $  1 10 
million,  while  retaining  about  $8  million 
as  his  total  operating  management  reserve. 

Furthermore,  he  would  alert  his  cus¬ 
tomer  to  the  possibility  that  if  all  goes  bad, 
the  project  could  cost  as  much  as  $128 
million.  The  customer  may  elect  to  take 
action  to  protect  his  funding  allocation  for 
this  worst  case.  The  analysis  will  reveal 
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the  high-risk  cost  elements  so  that  they  can 
receive  the  special  management  attention 
warranted. 

The  corresponding  Monte  Carlo 
probability  forecast  of  a  series  of  inter¬ 


dependent  schedule  elements  for  our 
hypothetical  project  might  be  similar  to 
that  in  Figure  4.  The  schedule  risk  is 
quantified  and  illustrated  for  all  to  see  and 
better  understand  the  uncertainties  of  the 


Figire  4.  A  Merit©  GarloScHedLte  Forecast  Genpletian  Date 
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project.  Based  on  this  analysis  the  project 
manager  would  take  action  necessary  to 
accommodate  both  the  possibilities  of  an 
early  delivery  or  a  late  delivery.  The 
Monte  Carlo  schedule  risk  analysis  may 
well  reveal  that  the  critical  path  does  not 
determine  the  most  likely  delivery  date  as 
commonly  assumed.  A  noncritical  path 
that  has  much  higher  risk  may  drive  de¬ 
livery.  Quantification  and  identification  of 
these  high-risk  paths  will  direct  and  help 
focus  management  attention  to  the  truly 
critical  program  elements.4 

Conclusion 


Cost  and  schedule  risk  mitigation  can 
best  be  done  if  the  risk  is  quantified.  Build¬ 
ing  detailed  cost  and  schedule  models  are 
always  formidable  tasks,  and  adding  the 


complexity  of  risk  estimates  can  be  a  sig¬ 
nificant  extra  effort.  But  with  the  simpli¬ 
fied,  pragmatic  approach  suggested  here, 
such  quantification  of  risk  is  a  practical 
and  productive  effort.  The  computational 
tools  and  the  software  to  support  the  meth¬ 
odology  suggested  are  currently  available. 
The  benefit  of  quantified  risk  is  best  illus¬ 
trated  by  the  wealth  of  management  in¬ 
formation  clearly  communicated  by  the 
previous  two  summary  forecast  charts. 
Forecasts,  made  for  projects  at  the  Naval 
Research  Laboratory,  such  as  the  interim 
control  module  (ICM)  for  the  NASA  space 
station,  have  provided  a  management  rec¬ 
ognition  and  comprehension  of  the  impact 
of  uncertainty  and  risk  associated  with 
schedule  and  cost  projections.  All  man¬ 
agers  will  be  well  served  by  using  such 
data. 
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Endnotes 


1 .  Cost  and  schedule  estimates  are  made 
up  of  many  independent  elements.  If 
each  element  is  planned  as  best  case — 
say  with  a  probability  of  achievement 
of  10% — then  the  probability  of 
achieving  best  case  for  a  two-element 
estimate  is  1%;  for  three  elements, 
.01%;  and  for  many  elements,  infini¬ 
tesimal.  In  effect,  it  is  zero. 

2.  No  attempt  will  be  made  in  this  ar¬ 
ticle  to  distinguish  between  risk  and 
uncertainty.  Risk  involves  uncertainty 
but  it  is  indeed  more.  For  purposes  of 
this  article  it  is  unimportant.  The  ef¬ 
fect  is  combined  into  one  statistical 
factor  we  call  “risk,”  which  will  be 
described  by  a  single  probability  dis¬ 
tribution  function. 


3.  The  difference  that  may  result  from  a 
Monte  Carlo  forecast  of  a  triangular 
PDF  and  a  forecast  using  a  corre¬ 
sponding  Beta  or  Weibull  PDF  is  most 
certainly  masked  by  the  accuracy  of 
their  estimated  parameters. 

4.  This  article  addresses  the  forecast  of 
possible  outcomes  of  cost  and  sched¬ 
ule  and  not  the  system  or  program  im¬ 
pact.  With  these  forecasts  managers 
can  anticipate  and  plan  for  possible 
impacts. 
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